Experiments with Μartmap: Effect of the Network Parameters on the Network Performance

Fuzzy ARTMAP (FAM) is currently considered as one of the premier neural network architectures in solving classification problems. One of the limitations of Fuzzy ARTMAP that has been extensively reported in the literature is the category proliferation problem. That is, Fuzzy ARTMAP has the tendency of increasing its network size as it is confronted with more and more data, especially if the data are noisy and/or overlapping. A modified version of Fuzzy ARTMAP, referred to as Safe μARTMAP, has been introduced in the literature by Gomez-Sanchez and his colleagues, in order to remedy the category proliferation problem. However, Safe μARTMAP’s performance depends on a number of network parameters. In this paper, we performed an exhaustive experimentation to find the best Safe μARTMAP network for a variety of problems (simulated and real problems), and compared this best performing Safe μARTMAP network with other best performing ART networks, including other ART networks that claim that resolve the category proliferation problem in Fuzzy ARTMAP. Finally, through this experimentation with Safe μARTMAP we were able to identify good default values for the Safe μARTMAP network parameters in a variety of problems. INTRODUCTION The Adaptive Resonance Theory (ART) was developed by Grossberg (1976). One of the most celebrated ART architectures is Fuzzy ARTMAP (Carpenter et al, 1992), which has been successfully used in the literature for solving a variety of classification problems. Some of the advantages that Fuzzy ARTMAP possesses is that it can solve arbitrarily complex classification problems, it converges quickly to a solution (within a few presentations of the list of the input/output patterns belonging to the training set), it has the ability to recognize novelty in the input patterns presented to it, it can operate in an on-line fashion (new input/output patterns can be learned by the system without retraining with the old input/output patterns), and it produces answers that can be explained with relative ease. One of the limitations of Fuzzy ARTMAP that has been extensively reported in the literature is the category proliferation problem. That is Fuzzy ARTMAP has the tendency of increasing its network size, as it is confronted with more and more data, especially if the data are noisy and/or overlapping. In this paper we focus our attention on one Fuzzy ARTMAP modification, called Safe μARTMAP, and introduced by Gomez-Sanchez, et al. (2001) that addresses this category proliferation problem. In particular, we perform an exhaustive experimentation to find the best μARTMAP network for a variety of problems (simulated data and real data), and we compared this best performing μARTMAP network with other best performing ART networks, such as Fuzzy ARTMAP (Carpenter, et al., 1992), Ellipsoidal ARTMAP (Anagnostopoulos, et al., 2001), Gaussian ARTMAP (Williamson, 1996 and 1997), and their semi-supervised versions (Anagnostopoulos, et al., 2003). Finally, through this experimentation we were able to define good default values for the μARTMAP network parameters, applicable to a variety of problems. μARTMAP ARCHITECTURE In this section, we provide a summarized explanation of μARTMAP, due to lack of space. More details about μARTMAP and Safe μARTMAP can be found in GomezSanchez’s papers. This section assumes that one is familiar with the ART architectures. As it is the case with other ART architectures that solve classification problems μARTMAP consists of three layers of nodes. The input layer, where the input patterns, designated as I, are applied, the output layer, where the corresponding outputs, designated as label(I), are applied, and the category representation layer, where compressed representations of the input patterns I are formed. The compressed representations in the category represenattion layer of μARTMAP are designated by aj w (the subscript index j designates the category). The weight vector aj w , referred to as template, has a hyper-box geometrical interpretation, such that the boundary of this hyperbox encloses all the input patterns that chose and were encoded by category j . The μARTMAP makes use of weight vectors of the form ) , , , , ( 1 ab N j ab jk ab j ab j b W W W ... ... = W . This weight vector emanates from a category j in the category representation layer and converges to all the nodes (Nb of them) in the output layer. Component ab jk W of this weight vector designates the number of times that category j was activated by an input pattern with corresponding output label k. The components of ab j W are used by μARTMAP to calculate the entropy of the categories formed, during the training phase of the network. The training phase of μARTMAP is succinctly described as follows (Steps 1-2). In all of the following equations, the notation | | ⋅ stands for the size of a vector and it is equal to the sum of its components, while the notation ∧ stands for the “fuzzy-min” of two vectors and it is defined to be the minimum, component-wise of these two vectors 1) (Learning Phase) Find the nearest category in the category representation layer of μARTMAP that resonates with the input patterns. That is, for each pattern I, the existing (committed) categories compete and the winner category is chosen to be the one that maximizes the following value (called bottom-up input): | | | | ) , , ( a j a j a j j T w w I w I + ∧ = α α (1) However, if the winner fails either of the following tests, it will be deactivated and the next winning category will be chosen and tested. a. Vigilance test: a a a j M ρ ≥ ∧ | | w I (2) where ρa is initialized as a predefined parameter ρ __ a and might increase in the offline evaluation phase according to (6). This test prevents the category from growing too large. b. Entropy test: max h hj ≤ (3) where ∑ =         − = = b N